Skip to main content

View container logs in production

Traces are sent to Axiom only after their context completes successfully. If the context never finishes - for example, if a thread hangs or the instance is terminated before it ends - the trace won’t be sent. In those cases we can view the container logs directly to see any unsent telemetry or system logs (like container termination)

View container logs in AWS​

Note: Only recently finished tasks are visible in ECS. For a more reliable view of recent activity or failures, check the logs in CloudWatch (the process should look quite similar).

To do this:

  1. Find a trace that might be relevant (for example, one that occurred a few seconds before the crash in the same session). Notice: it might be tricky!!! Because of the load balancer, events might happen near to each other but they will be in different containers. To look for all the containers , you might want to run Axiom query like this:
traces
| where * contains "SOMETHING_YOU_WANT_TO_SEARCH_FOR"
| project ['resource.cloud.container.id']
| distinct ['resource.cloud.container.id']

  1. Copy the container id of that trace from the field resource.container.id. For example: arn:aws:ecs:us-east-1:992382783210:task/backend/75b79ca2b9794001bacc72fb21ba9387.

  2. From that string, extract the the task id - 75b79ca2b9794001bacc72fb21ba9387.

  3. Then go to AWS Portal, and continue with the relevant profile -> Select the relevant region -> Search and navigate to ECS (Elastic Container Service) -> Select backend -> Click on the Tasks tab -> Search for the task id you extracted in the previous step, in our example 75b79ca2b9794001bacc72fb21ba9387.

Search for task id in ECS

  1. Click on the found task.

  2. Navigate to Logs tab.

  3. You are done! Now you can see & filter for all logs in this container - Good Luck!

View container logs in GCP​

  1. Find a trace that might be relevant (for example, one that occurred a few seconds before the crash in the same session). Notice: it might be tricky!!! Because of the load balancer, events might happen near to each other but they will be in different containers. To look for all the containers , you might want to run Axiom query like this:
traces
| where * contains "SOMETHING_YOU_WANT_TO_SEARCH_FOR"
| project ['resource.cloud.container.id']
| distinct ['resource.cloud.container.id']

  1. Copy the container id of that trace from the field resource.container.id. For example: 00147782968a84540c900fc030a5d7a94cbb0f97486fd5c536271c742098e974612e2eea49da8a5b99a78b4fd6a2db9efa806c9a857cd093ddfbe1ec7d29e2ea87c2f1a63700d8820a2d85c62165.

  2. Go to GCP Logs Explorer in the project for the relevant region. For example https://console.cloud.google.com/logs/query?project=backend-us-east4

  3. Search for all logs from the instance id in the relevant time range. Example basic query:

    labels.instanceId="[instance-id]"